All voice network service providers who want to introduce packet transport technology or any other technology (e.g., digital cellular) that increases connection delay beyond approximately 25 msec (round trip) need to deploy echo cancellers within their networks and these devices need to perform their function across a wide range of usage variation. The basic objective of a well-performing echo canceller (EC) is to quickly recognize an echo and then rapidly “converge” to cancel the echo by reducing it in level below the threshold of notice. Accordingly, it is very important to test an echo canceller (EC) prior to deployment in the network to ensure that it will perform to expectations.
A challenge in the area of echo canceller (EC) performance testing concerns what test signal(s) to use. Today, the standard EC test is based on International Telecommunication Union-Standardization Sector (ITU-T) Recommendations G.165 and G.168. ITU-T Recommendation G.165 recommended the use of band-limited (300-3400 Hz) white noise as the test signal. The updated, G.168 Recommendation defined and recommended a new test signal, the composite source signal (CSS) and the pass/fail criteria is based on results generated with this signal.
The CSS is a speech like signal in that it has a power density spectrum similar to that of speech, it is interrupted by gaps that simulate the pauses found in speech, and it simulates both voiced and unvoiced sounds. Like all good test signals, the CSS can also be specifically defined and, thus, constructed with fidelity in the different test labs. The CSS is generally regarded as a superior test signal to the white noise test signal that it replaces and performance seen with the CSS is assumed to be predictive of how the EC will perform in the presence of speech. However, there is ample evidence that shows that this prediction too often does not hold.
The existing “artificial” test signals used in the telecommunications industry for evaluating the performance of an echo canceller (EC) (i.e., those recommended by the ITU-T in Recommendations G.165 and G.168) frequently fail to predict how the echo canceller will perform in “real” uses (i.e., when the EC is working on actual speech signals). Specifically, testing an echo canceller with a CSS signal is not always predictive of its performance in the presence of speech. Consequently, pre-deployment performance testing of new EC designs does not provide the level of confidence desired in that the performance seen in the lab is not always the performance experienced in the field.
Since echo cancellers are deployed to control the echo of speech signals, the test signal of choice would seem to be a speech signal(s). It has been proposed to use a speech signal as the test signal, but there has been no agreement in the industry as to which speech signal to use. Speech signals vary widely from person to person and EC performance is sensitive to this variation.
Attempts to substitute speech signals in the G.168 testing reveal substantial performance variation driven by the specific speech sample in use. So it would appear that a sample of speech signals needs to be identified and used. This conclusion is not a satisfying one. G.168 EC testing is complicated enough with a single test signal in use. Trying to get general agreement to move to a sample of speech signals for testing is likely to be difficult if not impossible. Furthermore, to get lab-to-lab conformity, these signals would have to be shared since their independent reproduction is also not likely due to the complexity of real speech signals.
One approach to handling this complexity is to use multiple speech signals in an EC performance test and to score EC performance in terms of the results of a subjective mean-opinion-sore (MOS) test, where groups of test subjects listen to the processed speech samples and rate them in terms of their transmission quality based on the residual echo seen with the speech samples. The problem with this approach is that it does not provide the single, objective test signal needed to do routine testing and for setting objective performance requirements. Also, the MOS technique is rather expensive to implement and, under ideal conditions, can take days to conduct.
Accordingly, there is a need for an improved “objective” test signal for use in echo canceller performance evaluations. The currently used signals of band limited white noise and the CSS both have general utility but neither represent speech as well as they should. Thus, it would be desirable to provide an artificial test signal which generates performance that correlates highly with that observed with actual speech signals.